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Preface 


This manual is based on our study of LOFT debriefings at 
several U.S. airlines. The suggestions in this manual are derived 
from the data from that study, our subjective impressions, the 
experiences the LOFT instructors shared, and general literature 
on facilitation. Data and references to relevant literature from the 
study are available in the published report: LOFT Debriefings: 
An Analysis of Instructor Techniques and Crew Participation, 
by R.K. Dismukes, K.K. Jobe, and L.K. McDonnell (NASA 
Technical Memorandum 110442; March 1997). 

This material is presented as suggestions rather than rules 
because facilitation is very much a personal skill and each 
instructor must develop an approach with which he or she is 
comfortable. These suggestions provide a tool kit of techniques 
instructors may draw upon to develop their own style. 

This study was funded by the FAA’s Office of the Chief 
Scientist and Technical Advisor for Human Factors (AAR- 100). 
Eleana Edens was the program manager. 
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Facilitating LOS Debriefings: 

A Training Manual 

Lori K. McDonnell 1 , Kimberly K. Jobe 1 , R. Key Dismukes 2 

Ames Research Center 

SUMMARY 

This manual is a practical guide to help airline instructors effectively facilitate debriefings of Line 
Oriented Simulations (LOS). It is based on a recently completed study of Line Oriented Flight 
Training (LOFT) debriefings at several U.S. airlines. As a companion piece to the published report 
of that study (LOFT Debriefings: An Analysis of Instructor Techniques and Crew Participation, by 
R.K. Dismukes, K.K. Jobe, and L.K. McDonnell, NASA Technical Memorandum 110442, March 
1997), this manual presents specific facilitation tools instructors can use to achieve debriefing 
objectives. The approach of the manual is to be flexible so it can be tailored to the individual needs of 
each airline. Part One clarifies the purpose and objectives of facilitation in the LOS setting. Part Two 
provides recommendations for clarifying roles and expectations and presents a model for organizing 
discussion. Part Three suggests techniques for eliciting active crew participation and in-depth 
analysis and evaluation. Finally, in Part Four, these techniques are organized according to the 
facilitation model. Examples of how to effectively use the techniques are provided throughout, 
including strategies to try when the debriefing objectives are not being fully achieved. 

1 San Jose State University Foundation 

2 NASA Ames Research Center 


Part 1 . An Introduction to Facilitation 


How much crews learn in Line Oriented Simulations (LOS) 
and take back to the line hinges on the effectiveness of the LOS 
debriefing. The simulation itself is a busy and intense experience; 
thoughtful discussion afterwards is necessary so the crew can sort 
out and interpret what happened and why. As the instructor, you 
are expected to encourage the crewmembers to analyze their LOS 
performance on their own, rather than lecturing to them about what 
/ Keep the discussion they did right and wrong. This “crew-centered” approach 

crew-centered. emphasizes self-discovery and self-critique. The crew-centered 

approach also draws upon the crew’s professional experience and 
motivation to perform well to enhance learning. 


y Encourage crewmembers 
to participate actively. 


■/ Promote transfer of 
learning to the line. 


/ Use facilitation to meet 
debriefing objectives. 


The rationale for crew-centered debriefings is that adults learn 
and remember more when they participate actively and make their 
own analyses rather than when they listen passively to someone 
else’s. Active participation in the debriefing requires the 
crewmembers to process the information more deeply, enabling 
them to draw upon that information more readily and more 
effectively in a wide range of line situations. 

Another advantage of the crew-centered approach is that 
crewmembers who actively discuss Crew Resource Management 
(CRM) concepts and company procedures tend to “buy-in” more 
deeply than those who are only given lectures on proper 
procedures. Therefore, crewmembers who participate actively may 
be more likely to transfer learning from the LOS to the line. The 
goal is for crewmembers to develop the habit of analyzing their 
own CRM and technical performance following line operations, a 
practice which is still rare in civil operations. The LOS debriefing 
provides an opportunity for showing crews how to debrief and for 
illustrating die benefits of self-debriefing. 

Facilitation does have one distinct disadvantage: it is consider- 
ably slower than lecturing so less material can be covered in a 
short time. Some types of material, such as teaching the mechanics 
of a hydraulic system, are better suited to lecturing than 
facilitation. However, this type of instruction is not the primary 
purpose of the LOS debriefing; the primary purpose is to help 
crews analyze their performance and identify how they used CRM 
to manage all aspects of the LOS. (CRM is a method of managing 
all aspects of flight operations. Thus, discussion of CRM includes 
flight management issues such as teamwork, workload 
management, task management, and communication, as well as 
associated technical issues.) 

The facilitation techniques presented in this manual are 
intended to provide you, the instructor, with a set of tools you can 
use to effectively facilitate crew-centered debriefings. These tools 
supplement, rather than replace, the skills you already possess. 
Becoming skillful at facilitation requires practice, but once 
mastered it enables you to increase what crewmembers learn and 
take back to the line. 

Effective facilitation enables you to meet the following 
objectives of crew-centered debriefings: 

►F Crewmembers discuss issues directly with each other (rather 
than interacting solely with you, the instructor) and discover as 
much on their own as possible. 

*>- The crew thoroughly analyzes and evaluates what happened in 
the LOS, how they handled the situation, what went well, what 
could be improved, and how to improve it. 
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►>- The crew recognizes how CRM techniques helped them manage 
or could have helped them manage the situations they 
encountered in the LOS. 

*)- Crewmembers leave the session with a better understanding of 
how they can use CRM on the line to enhance safety and 
efficiency. 

►f The crew is encouraged to develop the habit of self-debriefing 
following line operations. 

Instruction vs. Facilitation 


/ Ensure crewmembers 
do most of the talking. 


/ Adapt the level of 
facilitation you use to 
the capabilities of 
each crew. 


V Exercise your dual 
role as instructor and 
facilitator. 


/ Reinforce good crew 
performance following 
crew analysis. 


In a crew-centered debriefing, the objective is for you, as the 
instructor, to facilitate crew discussion so that the crewmembers 
do most of the talking, participate proactively, discuss issues with 
each other, and thoroughly analyze the situations that confronted 
them as well as how they managed those situations. Some of the 
CRM literature espouses the ideal that crews debrief themselves 
and use the instructor as a resource. Realistically, most crews lack 
sufficient experience in analyzing the CRM aspects of cockpit 
operations to be able to conduct their own debriefings without 
assistance. Thus, as the instructor, you need to facilitate the crew’s 
analysis of their LOS performance and avoid centering the debrief- 
ing on your own perceptions. Crews vary considerably in how 
well they analyze what happened in the LOS and how well they 
respond to encouragement to participate actively; thus, you must 
adapt the level of facilitation to the capabilities of each crew. (The 
levels of facilitation are discussed in detail later in this section.) 

The concept of facilitation was originally derived from settings 
outside of aviation. In some of those settings the facilitator would 
act only as a moderator of discussion and lacked the technical 
expertise held by the group members. In the LOS setting, 
however, you play the dual role of facilitator and instructor. 
Through facilitation you enable the crew to figure things out for 
themselves; your expertise as an instructor enhances the crew’s 
understanding of the points they missed on their own. 

Facilitation does not require that you withhold your own 
perspectives. From ‘the back of the cab’ you often see things the 
crew does not notice and you can share your experience in how 
CRM can be used to manage the specific events that occurred in 
the LOS scenario. Since crews learn better when guided to self- 
discovery, you should use facilitation to promote that self- 
discovery in the crew and encourage the crewmembers to analyze 
their performance to the fullest extent possible. Once crew- 
members have completed their analysis, you can reinforce the 
things the crew did well. For the points crewmembers miss, you 
can provide the instruction necessary to ensure the training 
objectives are met. 
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Additionally, at times you may need to correct erroneous 
statements made by the crew and/or provide specific information the 
crewmembers lack. This is better done by direct instruction than 
through facilitation. You can either integrate brief instructional 
comments into the crew’s discussion or provide instruction as 
needed after crewmembers have finished their analysis. 

When guiding the crew’s analysis, you should avoid creating the 
impression of leading the crew to predetermined answers. The crew 
will not be motivated to participate unless you convey that the 
topics, issues, and perceptions they raise are important. 

What You Should Do to Facilitate the Debriefing 

*>- Set expectations for crew participation. 

Guide the session to the extent necessary to achieve the 
debriefing objectives. 

Adjust facilitation to the level needed to engage the crew to the 
maximum extent possible. 

►)- Draw out quiet crewmembers. 

►)- Ensure that all critical topics are covered. 

►>- Integrate instructional points as needed into the crew’s 
discussion. 

*>■ Reinforce positive aspects of the crew’s behavior. 

What You Should Avoid Doing 

•f Avoid lecturing and having the debriefing become an 
instructor-centered session. 

Long monologues or giving your own analysis and evaluation 
before allowing the crew to work issues out for themselves 
preempts the crew’s own analysis and participation. Avoiding 
lecturing does not mean you cannot teach, but you should teach 
through facilitation by integrating your own perspectives into the 
crew’s discussion. Rather than telling the crew what they did wrong 
during the LOS and how they can improve, try to get the crew to 
figure it out for themselves. If they discover what they need to work 
on by themselves, then they are much more likely to learn from their 
mistakes and carry that learning over to the line. 

Avoid giving your own analysis and evaluation 
before the crew has completed their analysis. 

Crews leam the most from their LOS experience when they 
make their own analyses. It is important that crewmembers leam to 
analyze and evaluate their own performance so that they can develop 
the habit of debriefing their performance following line operations. 
If you give your analysis before the crew does, the crew will feel 
less responsible for making their own analysis. When a second 
instructor is playing the role of a crewmember, he or she should 
participate as a crewmember but should wait and give his or her 
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analysis after the ‘real’ crewmembers have had the opportunity to 
analyze their performance as a crew. 

*>■ Avoid giving the impression that only your 

perceptions are important. 

You need to make it clear to the crew that you are interested in 
what they have to say in order to encourage them to participate 
actively and analyze and evaluate their performance in depth. 

►f Avoid interrupting crew discussion. 

Interrupting shows a lack of interest in the crew’s views and 
may disrupt their train of thought and hinder their process of self- 
discovery. Interruptions also undermine the crew’s sense of 
responsibility for making their own analysis and evaluation. Rather 
than interrupting a crewmember, try writing a note to yourself and 
then bring up your issue after the crewmember has completed his or 
her comment. (Interrupting may be necessary if a crewmember 
makes a statement contrary to your company’s SOP or the FARs.) 

Avoid interrogation; be positive when discussing 

problems. 

In order for the crew to be able to take responsibility for 
initiating topics and discussing issues interactively, you must work 
to keep the discussion crew-centered. Asking questions in a 
manner that resembles interrogation rather than guidance can hinder 
crew discussion, forcing the crew into a reactive, rather than a 
proactive, role. 

Avoid having a rigid agenda. 

Have the crew help develop the agenda to encourage them to 
identify areas in the LOS where they did well and/or need 
improvement. Keep the focus of the debriefing on topics introduced 
by the crew, but introduce your own points as necessary to expand 
on crew-initiated topics and to cover important issues not initiated by 
the crew. 

>> Avoid shortchanging high-performing crews by 

cutting their sessions short. 

Do not assume that because a crew did well there is nothing to 
talk about so a shorter session will suffice. Crews that perform well 
may not know why the LOS went well. Make sure that these crews 
recognize and discuss which CRM techniques resulted in their 
positive outcomes so the crewmembers can transfer this knowledge 
to other situations and when pairing with other crewmembers on the 
line. Good crews can learn from analyzing what they did well and 
why it worked. By analyzing their performance the crew can learn 
what strategies were effective and how to use those strategies on the 
line. Crewmembers are more likely to transfer effective performance 
to the line if it has been reinforced by discussion. 
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Levels of Facilitation 


Your role as the instructor in the debriefing session will vary as 
a function of the level of facilitation needed for each crew. Ideally, 
the crew will be able to analyze and evaluate their own performance 
and you will need to give only minimal guidance, thus utilizing 
high-level facilitation. Unfortunately, this rarely happens in the 
current training environment. Most crews lack the experience to 
participate at this level. Thus, you must adapt to the capabilities of 
each crew. Currently, most debriefing sessions are conducted at an 
intermediate level of facilitation. These sessions are largely driven 
by the instructor, who leads the crew to important topics and utilizes 
questions to evoke in-depth analysis and evaluation by the crew. 

The level at which facilitation is conducted is a broad continuum 
from high (most desirable) to low (least desirable). To illustrate this, 
we will discuss the high, intermediate, and low ranges in detail. Be 
aware that you may need to adjust the level of facilitation to whatever 
level is appropriate throughout the session. You should always 
/ Use the highest level of strive to work at the highest level of facilitation (i.e., the least inter- 
facilitation possible. ference on your part) at which each crew is able to respond. If you 

facilitate at too low of a level (i.e. more interference from you), the 
crew will be prevented from participating to their full capability. A 
sign of skillful facilitation is being able to recognize what level is 
✓ Adapt the level to appropriate for each crew, as well as being able to adapt to the vary- 

accommodate varying ing needs of individual crewmembers as the debriefing discussion 

needs throughout the shifts from one topic to another, 
session. 

If you consistently use facilitation to generate crew-centered 
debriefings, crews will gain experience as they go through recurrent 
LOS and they will participate at increasingly higher levels each year. 

High-Level Facilitation 

High-level facilitation is possible when the crew discovers and 
discusses important issues on their own with little guidance. 
Debriefing at this level provides the deepest and most enduring 
learning because the crewmembers guide their own self-discovery. 
For facilitating at this level, the crewmembers must be able to 
identify important topics and issues that arose during the LOS, set 
an agenda for discussing these issues, and critically analyze the 
situation they faced and evaluate how well they performed. 

At this level of facilitation, your role as the instructor is to 
inform the crew of the objectives for the debriefing, outline the 
debriefing process, and assist in guiding the discussion only when 
necessary. As the instructor, from your advantaged position as an 
experienced observer, you serve as a resource for the crew to help 
reinforce crew observations and supplement their perceptions. 
Although you retain responsibility for ensuring that the debriefing 
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objectives are met, you achieve this through general guidance rather 
than leading the discussion moment-to-moment. 

Intermediate-Level Facilitation 

When crewmembers are not as adept at conducting their own 
analysis and evaluation of the LOS, you must lead the discussion 
more directly. At the intermediate level of facilitation you may have 
to help the crew discover important issues and lessons by asking 
questions which lead them to specific issues and topics. You may 
also need to encourage the crew to analyze the situation and their 
performance in greater detail. At this level, crewmembers are able to 
achieve self-discovery through analysis and evaluation of their 
performance with your guidance. You may need to supplement crew 
analyses substantially, but should do so only after the crewmembers 
have completed their own analysis. 

Low-Level Facilitation 

Low-level facilitation does not imply inadequate facilitation on 
your part as the instructor; rather, it is the necessary and appropriate 
level to use when crews do not respond to higher levels. When 
crewmembers show little initiative and respond only superficially to 
your efforts to get them to analyze the LOS situations and evaluate 
their own performance, you may be forced to be more direct and to 
lead the discussion step by step. However, few crews are truly 
incapable of participating at a higher level. (Before retreating to this 
level of facilitation, be sure that you are not unwittingly 
discouraging crew participation by conveying the tone of an 
interrogation or by showing a lack of interest in what the 
crewmembers say.) At this level, self-discovery by the crew is 
limited but you should still use facilitation techniques to lead the 
crew to critical issues, appropriate solutions, and correct evaluation. 
At the end of the discussion of each topic you may have to explicitly 
summarize the nature of the problem encountered in the LOS and 
describe how it should be handled. When low-level facilitation is 
necessary, continue trying to elicit crew participation at a higher 
level. If it becomes necessary to instruct the crew on important 
points, confirm that they understand and agree, and continue to 
encourage them to initiate further discussion on the own. 
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Criteria for Effective Crew Participation 

In order to determine the appropriate level of facilitation to use, 
you need to be aware of how capable the crewmembers are of 
participating. The following criteria can be used to determine what 
level is appropriate for each crew at various times during the 
debriefing. If the crewmembers meet all of these criteria, high-level 
facilitation is best. If not, intermediate- or low-level facilitation may 
need to be used. 

*>• Crewmembers analyze in depth their LOS performance, 
discussing the situations they confronted, what they did to 
manage the situations, and why they made the decisions and 
performed the actions they did. 

•f Crewmembers evaluate in depth their LOS performance, 

discussing what went well, what did not go well, and how their 
performance could have been improved. 

•f Crewmembers discuss how CRM techniques helped them 
manage or could have helped them manage the situations they 
encountered in the LOS. 

Crewmembers address each other directly and interactively 
discuss the LOS rather than merely respond to your direction. 

Criteria for Effective Instructor Facilitation 


Your effectiveness in facilitating the debriefing can be measured 

according to the following criteria: 

You facilitate the crew discussion at the highest level possible, 
guiding the discussion only to the extent necessary to help the 
crew accomplish the objectives of the debriefing. 

►)- You modify the level of your facilitation to accommodate the 
crew’s needs throughout the debriefing. If the crew does not 
need guidance, you refrain from interfering with crew 
discussion. If the crew needs guidance, you use intermediate- 
level strategies to elicit continued and/or deeper discussion and 
revert to low-level facilitation only if necessary to ensure 
important lessons are learned. 

•f You ensure that the crewmembers analyze their performance in 
depth, identify CRM techniques that played or should have 
played a role in the LOS, and discuss the LOS with each other in 
an interactive way. 

You ensure that all critical topics are covered. 
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Part 2. Getting Started 


Clarifying Roles anc[ Expectations : The Introduction 

The Purpose of the Introduction 

►f To explicitly state how the crew should participate in the 
debriefing and how you, the instructor, will participate. 

To provide a persuasive rationale for crew-centered debriefing. 

Why Introductions are Important 

►>- The crew can’t be expected to participate in a certain way if they 
are not told how they are expected to participate. Individuals 
want and need to know what is expected of them. 

A good introduction encourages the crew to take responsibility 
for the debriefing session. 

^ Introductions have been shown to increase the amount and depth 
of crew participation. 

Important Points to Include in the Introduction 

Effective introductions should address all of the issues in the 
following four categories. 

The Instructor’s Role 

►*- Outline the process for the debriefing and help establish an 
agenda. 

>f Help the crew identify topics and facilitate crew discussion as 
necessary. 

Act as a resource for CRM and technical issues. 

Keep discussion crew-centered rather than instructor-centered. 
►>- Ensure that the training objectives are met. 

The Crew’s Responsibilities 

►>- Raise issues and initiate discussion. 

+ Discuss issues directly with each other rather than only with 
the instructor. 

Discuss CRM techniques used during the LOS and other 
techniques which could have been used profitably. 

Critically analyze the situations that were encountered and how 
they attempted to manage those situations. 

*)■ Evaluate how things turned out and why. 

>-)- Discuss what, if anything, they would do differently in the 
future. 


/ Clarify your role as 
instructor. 


y Detail your expectations 
for crew participation. 


9 



/ Provide a persuasive 
rationale for the 
debriefing being 
crew-centered. 


/ Tell the crew how long 
the session will last. 


/ Explain the format the 
debriefing will follow. 


The Rationale for Using Crew-Centered 
Debriefing 

•f Crews leam better through self-discovery and self-analysis 
than by lecture. 

•f It draws upon crews’ professional expertise and motivation to 
perform well. 

A- It helps the instructor understand the crew’s performance. 

The Expected Length and Format of the 
Debriefing 

Tell the crew how long the session will last (so they will not be 
motivated to keep quiet in hopes of a short session). A 
thorough crew-centered debriefing can usually be accomplished 
in an hour. 

Tell crews explicitly what format will be used for the 
debriefing (e.g., setting an agenda of topics to discuss, using 
videos to introduce and supplement discussion, using the 
model presented in this manual to keep focused on important 
topics). 


A Sample Introduction 

“How much you leam from the LOS depends on discussing it afterwards: Analyzing the 
situations that confronted you; analyzing what you did to manage those situations, evaluating how 
well it worked out, and identifying anything you might want to do differently. It is very important 
that you make this analysis on your own. We all know how much more we leam when we work 
something out ourselves than when we listen to a lecture about it, especially if it concerns our own 
performance. So, if you are expecting a lecture from me, forget it! 

I realize that debriefing yourselves may be new to you, so I will act as a guide and help you get 
started. I encourage you, however, to take the initiative instead of waiting for me to raise issues. 
When you see the video, please analyze what happened, what you did to manage the situation, and 
why you did it that way. What was the most difficult aspect and why? What CRM issues were 
involved and what CRM techniques did you use? We are not looking for a scripted solution here, 
because every situation on the line is a little different. The point of this exercise is for you to 
develop the habit of analyzing line situations and determining for yourselves how to best use all of 
your resources to manage those situations. 

I expect you to do most of the talking, raise questions about what was going on, identify 
issues, and volunteer your perspectives. You can use the agenda of topics we develop to structure 
your discussion. For each topic you need to go beyond just describing what happened and what 
you did. You need to dig deep, to analyze why it happened that way, what management techniques 
seem to work best, and what you might have done differently. This works best when you discuss 
the issues with each other and reach your own conclusions about the factors that contribute to 
successful operations. 

This session will last about (x) minutes. I have marked several places on the video that 
illustrate interesting aspects of the flight. But, before we start looking at those video segments, 
what portions of the flight do you feel are most important to discuss?” 
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Debriefing Formal 


Developing an Agenda for the Discussion 


/ Help the crew develop 
an agenda and identify 
issues for discussion. 


V Ensure that all critical 
issues are covered. 


A primary debriefing objective is for discussion to be crew- 
centered. As the instructor, you play a critical role in helping the 
crew develop a structure for their discussion and in ensuring that 
all important issues that arose during the LOS are covered. It is 
important to get the crewmembers to actively participate in 
developing the agenda so they can develop the skill of identifying 
issues that arise in line operations. 

Ask the crew to identify any topics, situations, or performance 
issues they would like to discuss. Remind them to include aspects 
of their performance that worked well, not just aspects that might 
be improved. 

The issues you noted during the LOS are also an important part 
of the debriefing agenda. As the instructor, you need to ensure that 
all critical issues are covered. You can introduce issues not 
identified by the crew at an appropriate point during the debriefing. 
(A good way to improve the ability of crews to identify issues for 
discussion is to suggest during the pre-LOS briefing that they 
make note of issues they want to discuss in the debriefing.) 

Typically, it is easiest to discuss phases of the flight in 
chronological order so videotaped segments of the LOS performance 
can be incorporated into the discussion without requiring extensive 
searching on the VCR. However, this is largely a matter of personal 
preference and some instructors prefer to start the discussion 
immediately when crewmembers suggest a topic in order to 
reinforce the crew’s initiative. One advantage of showing video 
segments related to each topic is that after viewing their performance 
objectively, crewmembers may be better able to analyze and evaluate 
their performance in depth. 
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Part 3. Facilitation Techniques 


Part of your role as the instructor is to ensure that the crew- 
members analyze and evaluate LOS situations and their performance 
in depth. One of the greatest difficulties you may encounter is getting 
the crew to go beyond a superficial description of LOS events. The 
objective is to encourage crewmembers to not just discuss what 
happened — they should discuss why they did what they did to 
manage the situation and why the situation turned out the way it did. 
Discussing why helps crewmembers better understand what 
knowledge and information played a role in their decision-making 
process, which in turn can help them learn to make better decisions in 
the future. 

This section will demonstrate how to effectively use the 
techniques that are-essential to successful facilitation. The next 
section (Part 4) will show how to integrate these techniques utilizing 
the C-A-L model. 


Questions 


y Use questions to promote 
in-depth crew participation. 


Questions are useful to guide the crew to important topics and 
encourage them to analyze and evaluate their LOS performance in- 
depth. Although questions are one of your most useful tools, you 
should be wary of dominating the discussion with your own 
questions. To the extent necessary, you can use questions to help 
the crewmembers identify issues and follow up on topics, and to 
encourage them to thoroughly analyze their performance through 
direct discussion with each other. Ideally, questions should be 
phrased to require more than a simple “yes” or “no” answer; 
questions should push the crew to analyze their performance at a 
deeper level. 


Note: The examples used in this section were taken from actual debriefings. The 
participants ’ titles have been substituted for names: 

IP = Instructor Pilot FE = Flight Engineer 

CA = Captain FEI = Instructor acting as Flight Engineer 

FO = First Officer 
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Set the Scene and Ask for Crew Reaction 

A good way to start a discussion of a LOS event is to show the 
video segment and/or describe the situation and then ask an open- 
ended question: “What went well (or not so well) there?” or “What 
lessons can be learned from this?” 

If the crew does not respond, try again with a more focused 
question: “Is there anything you would do differently if you had it to 
do over?” or “Did you see anything in the video that was not SOP?” 


Lead the Crew to Topics 

If the crewmembers seem unsure of what to discuss next, or if 
they overlook an important aspect of a situation, you may need to 
ask a question to focus their attention on some topic or aspect. 
However, be careful to phrase the question so it is not easy to 
answer with a simple yes or no and so it does not leave the crew 
with nothing to say. 

Effective: 

IP: What else about how you handled that emergency 

situation can you discuss? 

CA: Well now that you mention it I guess we did get a little 
bit rushed there and we didn’t communicate as well as 
we could have. 

Ineffective: 

IP: Did you get a little rushed at that point, making 

communication a little more difficult? 

CA: Yeah, we did. 


Deepen the Discussion 

Once the crew has begun to discuss a topic, you may need to 
encourage them to explore in more depth what happened. Ask 
questions that require the crewmembers to explore their thoughts 
and actions in the LOS. Avoid asking questions that give the answer 
or leave little for them to say. 

Effective: 

IP: Was there anything that made you uncomfortable 

during that first leg? 

FO: Well, yeah, I guess the R-Nav approach did. I wasn’t 
comfortable with it because I didn’t know we could 
shoot an R-Nav approach in there; I thought we were 
supposed to shoot a VOR. 

— or — 

(continued on next page) 
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IP: Was there anything that made you uncomfortable 

during that first leg? 

FO: Uh. . .not that I can think of. 

IP: What about the R-NAV approach? 

FO: Well, yeah, I guess the R-Nav approach did. I wasn’t 
comfortable with it because I didn’t know we could 
shoot an R-Nav approach in there; I thought we were 
supposed to shoot a VOR. 

Ineffective: 

IP: You seemed a little uncomfortable with that R-NAV 

approach. Were you? 

FO: Well, yeah, I guess I was. 


Follow-Up on Crew Topics 

It is important to follow-up on topics initiated by the 
crewmembers and to encourage the crew to explore their topics 
fully. Following up on crew topics demonstrates to the crew that 
their thoughts and ideas are important and reinforces their 
initiative. To obtain greater depth and detail on topics initiated by 
/ Ask questions that begin the crewmembers, ask questions that begin with what, how, and 
with what, how, or why to w hy. For instance, in the first example below, the more-skillful 

encourage deeper instructor follows up and gets the CA to comment on the FE’s 

observation while the less-effective instructor moves on to another 
topic. In the second example, a more in-depth answer is elicited by 
the more-effective instructor. 

Effective: 

FE: I think I should have just taken care of that for C A; 
tried to get that system back. 

IP: Let’s talk about that. How did you feel about it, CA? 

CA: Well, at that particular point I knew we had to stay 
away from the rocks, we had isolated the hydraulic 
system, and I was just not comfortable giving up 
control of the airplane to FO. I just wanted to fly out of 
there. I had B system, and it’s a lot more powerful than 
the standby A system, so I really didn’t need that 
system back. 

Ineffective: 

FE: I think I should have just taken care of that for CA; 

tried to get that system back. 

IP: I agree. Now, what’s next on the list? 
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Effective: 

FO: I felt pretty comfortable on the single engine approach. 
IP: Yeah? Why? What made you feel comfortable? 

FO: [describes in detail why he thought he did so well] 

Ineffective: 

FO: I felt pretty comfortable on the single engine approach. 
IP: Yeah, I thought you handled it well, too. 


Turn Crew Questions and Comments Back to Them 

If a crewmember makes a comment or asks a question, avoid 
using this as a springboard for presenting your own perspectives. 
Instead, turn the topic back to the crew for discussion. Continue to 
redirect their questions and comments to encourage the crew to 
/ Encourage the crew to work out answers for themselves. Your questions are a powerful 
discover their own tool for blending your observations into the discussion without 

answers. dominating. After the crewmembers have addressed everything 

they can, you can add teaching points they have not yet 
discovered. 



Effective: 

FO: I don’t know what happened there. Did I overrotate? 
IP: Did you overrotate? 

FO: Well ‘I started going through it at about twenty 
degrees. 

CA: Well, we may have gone over twenty just a little bit. 
IP: Why don’t you talk about that? 


I Effective: 

CA : How did you think we reacted as a crew to that 
compound emergency? 

IP: r 11 save my remarks for the end. Each of you tell us 

how you tlunk you reacted as a crew to the emergency. 


Get Crewmembers to Actively Participate 


/ Make sure that all 
crewmembers are fully 
drawn into the discussion. 


It is important to ensure that all members of the crew actively 
participate in the debriefing so they can all experience the higher 
level of learning that results from taking an active role in the 
learning process. Also, the crew cannot fully analyze and 
understand their performance as a team without each member’s 
perspective. The following techniques can be used to promote 
active participation by all crewmembers. 
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Drawing Out a Quiet Crewmember 

If one crewmember does not volunteer comments and responds 
only minimally to questions, you can use several techniques to 
encourage his or her participation: 

Ask the quiet crewmember to discuss how effectively 

the crew performed in the LOS. 

It is relatively easy to respond to this type of question because 
there is no right or wrong answer and all crewmembers are likely to 
have an opinion. 

Effective: 

• “FE, how effectively do you think you performed as a crew 
in resolving the TCAS alert incident?” 



Re-direct a question to the quiet crewmember. 

If your questions are predominantly answered by the same 
crewmember, repeat the question after the dominant crewmember 
has responded so that the quiet crewmember will be required to 
participate and communicate his or her opinions and ideas. 

Effective: 

IP: How did you like that first leg? What did you like 

about it? 

FO: [describes what he feels went well on the first leg] 

IP: CA, what did you like about the first leg? 

CA: [describes the things she thinks went well] 



•f Ask the quiet crewmember to expand on what another 
crewmember said. 

If a crewmember sits silently while another does all of the 
talki ng, it may be helpful to ask the silent one to expand on what 
was said by discussing a new aspect of the topic. This technique 
will make it easier for the quiet member to find something to say. 

Effective: 

• “FO, how do you think, as a crew, you could have handled 
the situation the CA was discussing?” 
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Drawing Out an Entire Crew 

Sometimes all or most of the crewmembers are unresponsive 
and getting them to participate seems like pulling teeth. When this 
y Avoid unwittingly hindering happens, you should first review your own participation to ensure 
crew participation. that you are not unwittingly sending covert messages that 

discourage crew participation. To instigate crew participation, it 
may be helpful to start with the more junior members so they will 
not be put in the position of potentially disagreeing with the CA 
early in the session. The next section suggests specific ways to 
stimulate participation when crews do not immediately respond. 

Troubleshooting: When the Crew Does Not 
Respond to Questions 

If crewmembers do not immediately respond to questions, the 
following techniques may be helpful in eliciting responses. Start 
with the highest level of facilitation and progressively work down 
only through the levels necessary so that facilitation stays at the 
highest level possible. 


High-Level Facilitation 

•f Use silence/pauses to elicit crew responses. 

Pause at least three to four seconds after asking a question 
y Don’t answer for the crew rather than immediately answering for the crew. Most 

when they don’t crewmembers will feel motivated to say something to end the 

immediately respond. uncomfortable silence (see the following section: Use of Silence). 



Effective: 

IP: Has there ever been a flight where errors didn’t occur? 

CA: No. 

IP: And what do we want to have happen when errors do 

occur? 

[IP waits 8 seconds, 

CA: We want somebody to say something and correct the 
error. 

IP: Exactly. 

Ineffective: 

IP: Has there ever been a flight where errors didn’t occur? 

CA: No. 

IP: And what do we want to have happen when errors do 

occur? 

[IP becomes uncomfortable when crew doesn ’t answer right 
away and answers for them.] 

IP: We want someone to notice that error, don’t we? 
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Intermediate-Level Facilitation 

+ Reword questions rather than give answers. 

When crewmembers do not answer a question after five to ten 
seconds of silence, it may be necessary to rephrase the question to 
make the intent of the question clearer or to ask a more specific 
question. If they still do not answer, again pause to allow them 
time to formulate a thoughtful response. Rephrasing the question 
may give the crewmembers the cue they need to spark a response, 
as demonstrated in the examples which follow. By being 
persistent and patient, you reinforce that the crewmembers are 
responsible for their own learning and that they are not going to 
get away with passively sitting by and waiting to be told the 
answers. When you answer for the crew or rephrase questions in 
a way that leaves little for the crew to say, you inhibit in-depth 
crew participation and hinder crew learning. 

Effective: 

IP: How did you manage the situation that arose? 

[The crew does not respond, so the IP asks a more specific 

question.] 

IP: What CRM techniques did you use to resolve the 

situation? 

CA: I created time by letting the FO fly the plane while I ran 
the emergency checklist. 

Ineffective: 

IP: How did you manage the situation that arose? 

[The crew does not respond, so the IP answers for them.] 
IP: You created time, didn’t you? 


Effective: 

FO: I don’t like the idea of having to roll the trim all the 
way down and then forgetting about it. 

IP: I agree. What do you think they could do to make that 

better? 

FO: Uh... [trails off] 

[Rather than answering for the crew, the IP rephrases the 

question to clarify what was not understood .] 

IP: What could you do to take care of that if you were 

redesigning the checklist? 

FO: I would reorganize the section on de-icing and have 
another after-start checklist again because that’s where 
the confusion is. That’s where the flow is broken up. 

(continued on next page) 
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Ineffective: 

FO: I don’t like the idea of having to roll the trim all the 
way down and then forgetting about it. 

IP: I agree. What do you think they could do to make that 

better 4 ? 

FO: Uh. . . [ trails off] 

[IP rephrases the question, but new question leaves little for 

crew to say.] 

IP: Couldn’t they reorganize the checklist so that item 

appeared sooner? 

FO: Uh, yeah, I guess so. 



Effective: 

If the crew does not respond to the question “What was the 
weak part of your VI cut?” and the problem was that they 
were trying to do a checklist and struggle with the airplane 
when they could have had the autopilot on, rather than 
pointing that out for them, ask a more specific question. 

• “What other resource could you have been utilizing at the 
time to help in that situation?” 



Effective: 

If the crew does not respond to the question “Why didn’t 
you finish the checklist?” and the reason they didn’t finish 
was that they were distracted by a warning light, rather than 
saying it for them, ask a more specific question. 

• “Was there something that distracted you during the 
checklist?” 


Low-Level Facilitation 

4- Answer for the crew, but confirm that they 

understand and agree. 

If the above facilitation techniques still do not elicit a response 
from the crew, or if the response given lacks depth, it may be 
necessary to briefly instruct the crew to ensure that they learn from 
the situation so they can apply the lesson on the line. If it becomes 
necessary to instruct the crew on certain issues, be sure to verify that 
they understand and agree because if they don’t the opportunity for 
learning may be lost. It is important to continue trying to involve the 
crew as much as possible. Once you have shared your knowledge 
with the crew and elicited their agreement, try to return to a higher 
level of facilitation by asking for the crew’s opinion, asking them to 
expand on the issue, or asking them to give an example of what you 
discussed from their line experience. 
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Use of Silence 


/ Use silence to elicit 
thoughtful crew 
responses. 


Silence can be a very useful tool to elicit thoughtful crew 
responses. Most people are uncomfortable with silence in a group 
setting and are more likely to make some kind of response the 
longer the silence lasts. Without realizing it, instructors often allow 
only a second or two for a response to their question before either 
calling on someone else or giving the answer. However, a second 
is generally not long enough to formulate a thoughtful response, 
especially to complex questions. 


Benefits of Using Silence 

Studies show that waiting three to four seconds after asking a 
question greatly improves the number and quality of responses. The 
specific benefits of pausing at least three seconds include: 

crewmembers produce longer and more confident responses. 

*>■ crewmembers are more likely to volunteer unsolicited responses. 

crewmembers interact more with each other. 

►)- crewmembers ask more questions. 

+ less articulate crewmembers participate more. 

What to Do during Silence 

In order to minimize the discomfort often associated with even 
brief periods of silence, it is important to behave in a way that makes 
the crew feel at ease. 

Look relaxed, not anxious or impatient. 

For the crew to feel comfortable with silence, it is important that 
you appear at ease. Appearing tense and anxious will give the crew 
clear non-verbal cues that silence is not appreciated. 

Sit back. 

Sitting on the edge of your seat may be interpreted by the crew 
as a sign of impatience. Sitting back in your seat, however, helps 
demonstrate that you are comfortable allowing periods of silence for 
the crew to think before they respond. 

Smile. 

S milin g conveys reassurance and acceptance, which can help put 
the crew at ease. 
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Strategies for Using Silence 

Silence can be used throughout the session to encourage crew 
participation, not only after asking a question but also after crew 
comments. Using silence is a high-level facilitation strategy — if 
silence alone does not elicit crew discussion, try the next level of 
facilitation. 

High-Level Facilitation 

►f Pause after asking questions to allow crewmembers 
to formulate thoughtful responses. 

When crewmembers do not immediately respond, be patient; 
remain silent rather than answer for them. 

I Effective: 

IP: How did you feel things were going on the first leg 

overall, FO? 

FO: Uh. . . [ trails off] 

[IP waits 6 seconds, 

FO: I felt a little disorganized pushing off of twenty-three 
and ta xiin g out, and doing all of that and then having to 
de-ice. That breaks your flow because you don’t put 
the flaps down. So it just seemed backwards to me. 

Ineffective: 

IP: How did you feel things were going on the first leg 

overall, FO? 

FO: Uh. . . [ trails off] 

[IP answers for him. ] 

IP: You felt a little uncomfortable, didn’t you? 

Pause after crew comments to allow crewmembers to 
resume discussion. 

When there is a lull in crew discussion, be patient and wait for 
the crew to say more rather than immediately adding your own input. 

I Effective: 

IP: FO, what do you think about what CA just said? 

FO: I think we all agreed it was a thousand. I was flying 
and still listening to what they were saying and it 
sounded okay to me. 

[IP pauses 8 seconds] 

FE: I think that things worked out pretty well because when 
FO was flying he was doing a good job and we didn’t 
have to worry about that part. 

CA: Yeah, I agree with you FE. I was comfortable with 
FO’s flying the airplane even when I was out of the 
loop because he was always ahead of the airplane. 
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Intermediate-Level Facilitation 

(Also see Troubleshooting in the previous section.) 

Ask crew to comment or elaborate on what was just said. 

If the crew does not resume discussion after a period of silence, 
encourage further discussion by asking them to elaborate on their 
previous discussion 



Effective: 

IP: FO, what do you think about what CA just said? 

FO: I think we all agreed it was a thousand. I was flying 
and still listening to what they were saying and it 
sounded okay to me. 

[IP pauses 8 seconds, then asks crew to elaborate.] 

IP: FE, how do you think this situation turned out? 

FE: I think that things worked out pretty well because when 
[FO] was flying he was doing a good job and we 
didn’t have to worry about that part. 


Low-Level Facilitation 

(Also see Troubleshooting in the previous section.) 

*>- Reinforce what the crew said. 

If the crew does not resume discussion after a lengthy silence 
and cannot elaborate further, use the opportunity to provide the crew 
with positive feedback on the issue they were discussing. Not only 
will the feedback reinforce the issue at hand, it will also reinforce 
crew discussion in general. 


Active Usteninci 


Good listening skills are as important to an instructor as good 
speaking skills. Active listening is one of the most useful tools you 
can use to encourage continued participation; it shows the crew that 
you are listening and paying attention to them and that you 
understand what they are saying, which in turn encourages them to 
keep talking. Active listening ranges from simple non-verbal 
gestures to expanding on crew comments. In all cases, the objective 
is to encourage the crew to keep talking and, in the more complex 
forms, to encourage deeper discussion of the issues. The following 
are examples of the various levels of active listening, in order from 
the simplest to the most complex. 

*>■ Non-verbal. 

Nod, smile, make eye contact, sit forward in your chair, or 
otherwise indicate that you are interested in what is being said. 


23 



+ Short interjections. 

Interject with “Yes?”, ”Uh-huh...’\ “I see...”, etc., to indicate 
that you are following what they are saying and to encourage them 
to say more. 

►F Echoing. 

Repeat part of what the speaker said as a question directed back 
to the crew (e.g., “So you weren’t sure if you were cleared for 
takeoff?”). 

Reflecting. 

Repeat what the speaker said in different words while retaining 
the same meaning (e.g., if speaker says “We didn’t use good 
communication” you might respond with “You didn’t let each other 
know what you were doing?”). 

►F Expanding. 

Expand on what the speaker said by implying more than the 
speaker intended (e.g., if speaker says “We didn’t use good 
communication” you might respond with “So if you had 
communicated better you could have avoided getting overloaded?”). 

With all of these active listening techniques, the most important 
aspect is the tone you convey to the crew. It must be apparent from 
your tone that you are genuinely interested in what they have to say. 
Furthermore, you must convey that you are interested in their 
perspectives, not just in leading them to predetermined answers. 


Use of Video 


The videotape of the LOS session is a useful tool for enhancing 
the debriefing. Relevant events need to be marked on the tape during 
the LOS session so you can draw the crew’s attention to important 
points for discussion. 

Benefits of Using Video 

An important contribution of videos may be to remind crews 
what transpired at critical segments during the LOS because it is 
often difficult for participants to remember specific details of the 
typically long and stressful sessions. Videos enable crews to review 
their actions and then examine and gain insight into their behavior 
with the detachment of observers. Videos can also encourage self- 
assessment by helping crewmembers identify behaviors that were 
either helpful or harmful to the outcome. The realistic feedback that 
videos provide can make it easier for you to involve the entire crew 
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in a discussion of team performance, thus enhancing the overall 
potential of the training session. 

Techniques for Using Video 

*)- Index important events. 

From your experience with the LOS scenario you will be aware 
of critical events that will challenge the crew. Index those events on 
the videotape during the LOS and make notes to remind yourself of 
important points to discuss. Also, index noteworthy examples of 
good and poor crew performance. 

F- Do not show a video segment unless you intend to 
discuss it. 

The purpose of showing the video is to enhance crew analysis 
by refreshing their memories and giving them the opportunity to 
objectively evaluate their performance. If you show a video segment 
without discussing it, you waste both time and the opportunity for 
discussion. 

F Avoid showing a large number of segments or very 
long segments. 

In a one-hour session, 3-6 segments are usually all you will 
have time to adequately discuss. Try to select these 3-6 short 
segments to illustrate some major aspects of crew performance 
(good or bad) rather than showing an entire phase of flight (e.g., 
from top of descent to touchdown). For aspects of a crew’s 
performance that cannot be clearly illustrated by the video, it is best 
to steer the discussion to these topics without introducing them with 
a video segment. 

*F Be proficient in the use of the video equipment. 

Be very familiar with the video equipment so you can quickly 
find the events marked on the tape. Use the automatic index marker 
to find events — playing the video in fast forward while searching for 
segments wastes time and distracts the crew. 

F Introduce each video segment. 

Set the scene by describing the situation and where in the flight 
the segment to be discussed occurs. You may also find it useful to 
remind the crew that they will be expected to analyze what they see. 

Effective: 

“This segment starts just as you were lifting off from LAX. 

I want you to watch what happens — think about your 
performance and what you were thinking as you began to 
realize something was not right so after the video you that 
can analyze what happened.” 

Ineffective: 

“Okay, here’s something I wanted you to see.” 
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*>■ Pause for comments. 

Pause the video when you or a crewmember makes a comment 
so the comment can be heard and understood. Crewmembers will 
feel that you are genuinely interested in what they have to say if you 
pause to give them your undivided attention. 

Effective: 

FQ: You know, another thing that concerned me (IP stops 
video so FO ’s comment can be heard ] was how we 
turned directly in to intercept the localizer and it 
happened quicker than I expected. 

Ineffective: 

FO: You know, another thing that concerned me was how 
we. . .[IP does not stop video, so remainder of FO ’s 
statement can ’t be heard over the VCR] 

+ Pause to discuss important issues. 

Pause the video to call attention to and elicit crew discussion of a 
specific aspect of crew performance shown on the video. For 
example, rather than talking during the video and forcing the crew to 
compete with the noise, stop the video and encourage crew 
discussion. 

I Effective: 

• “Okay, let’s just stop here. How did you communicate to 
ATC what you needed from them with regard to the 
hydraulic problem?” 

• “All right, let’s just pause it right here for a second. What 
did you just do there?” 
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Part 4. The C-A-L Model in Action 


The C-A-L Model provides a way to structure the debriefing. 
The first section, CRM, suggests strategies for helping the crew 
focus on CRM techniques that played a role in the LOS. The second 
section, Analysis and Evaluation, shows how to guide crews to 
identify and evaluate aspects of their performance that went well or 
could use improvement, including analysis of why the crew did 
what they did and why things turned out the way they did. The third 
section, Line Operations, provides a structure for helping the crew 
explore how they can apply what they learned from their analysis of 
the LOS to line operations. 

A good way to organize debriefing each segment of the LOS is 
to show the appropriate video segment and then use the components 
of the C-A-L Model to guide the discussion. 


C: CRM - Applying the Company Model 

The major purpose of LOS is to give crews a chance to practice 
using CRM concepts and techniques in realistic flight scenarios. 
Typically, crewmembers are much better prepared to talk about the 
purely technical aspects of a flight situation than the CRM aspects so 
they may need to be guided toward discussion of CRM issues 
pertinent to their performance in the LOS. Rather than discussing 
CRM as a set of abstractions, you can help crews the most by 
guiding them to consider how specific CRM techniques can be used 
to manage various flight situations. 
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Focusing on CRM 

You can use several techniques to bring CRM into the crew’s 
discussion in a concrete, relevant way. 

A- Refer to the posted CRM concepts. 

Each company teaches a specific framework for CRM concepts 
that reflects the company’s philosophy. Posting this conceptual 
framework on wallboards and referring to it during the discussion 
can help the crew relate CRM concepts and techniques to specific 
operational situations. 

I Effective: 

IP: Which of these crew effectiveness markers do you 

think you used to resolve that de-icing report problem? 
FO: We used ‘Inquiry and assertiveness’ and ‘Vigilance’ 
making sure everything was okay. 



Effective: 

IP: What CRM technique do you wish you might have 

used a little better? 

CA: ‘Anticipate required actions and workload distribution.’ 
That was a tough one for us because we were both 
very busy and there’s not much he can do except fly 
the airplane if he’s flying. We’re supposed to each 
follow along on these checklists, and I think that’s 
something we could have been better at. 

IP: What techniques can you use to manage workload 

better in the fiiture? 


I Effective: 

[The CA tells the FO to askATCfor a holding pattern in 
order to have time to do a manual gear and flap extension 
during a video segment. The instructor stops the video, 
points to the posted CRM markers and asks questions.] 
IP: What CRM principle is that an example of? 

CA: Create time. 

IP: And how did you create time? 
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•f Use CRM-specific questions. 

Get the crew to explore specific CRM issues and techniques that 
presented themselves during the LOS. 

Effective: 

• “What CRM techniques might have helped in this particular 
situation?” 

• “What specific CRM techniques did you use that resulted in 
your successful outcome?” 

• “What kinds of challenges did the LOS present in terms of 
how to manage your time?” 

• “Looking at your workload management, what CRM 
techniques did you use in this LOS that you learned in 
ground school?” 



Use guiding questions. 

Guiding questions can be used to lead the crew to more specific 
and in-depth aspects of their LOS performance. In the following 
example the instructor encourages the crew to discuss specific aspects 
of teamwork that occurred during the LOS. 

Effective: 

IP: Did you feel included? 

FO: Yes. 

IP: What did the CA do to make you feel included? 

— or — 

IP: Did you feel included? 

FO: Not really. 

IP: What could the CA have done to make you feel 

included? 



In the next example, the instructor facilitates crew discussion 
of specific aspects of workload management that occurred during 
the LOS. 



Effective: 

IP: Did you feel rushed or overworked or distracted perhaps? 

FO: Yeah, I felt like I wasn’t ready to turn in. 

IP: But you turned in anyway? 

CA: Yeah, I was ready. I didn’t ask you [FO] if you were ready 
though. 

IP: You weren’t ready, FO? And could you have said 

something to the CA about that? 

FO: I could have, but I usually don’t say anything as a co-pilot 
unless I think I’m in dire danger. 

IP: What would you have said if the FO had said “I don’t feel 

ready”? 

CA: I would have said “Let’s take another turn in holding” then. 
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Reinforcing the Utilization of CRM through Crew 

Interaction 

Instructors and crews too often fall into a pattern of discussion 
that centers entirely on the instructor: The instructor asks a 
question, a crewmember responds, the instructor comments, and 
the cycle repeats. The crew will benefit in several ways, however, 
if you can get them to discuss their performance in the LOS 
y Encourage crewmembers directly with each other. Interactive discussion between crew- 
to address each other members during the debriefing allows them to practice CRM skills 

directly. such as co mmuni cation and problem solving. It also leads them 

toward the ultimate goal of being able to debrief themselves in line 
operations. You can counter the crewmembers’ natural tendency to 
direct their comments to you by using the following techniques. 

Ask crewmembers to discuss how they were affected 

by each other’s actions. 

It is important that crewmembers understand how their actions 
effect each other. By openly discussing these issues, crewmembers 
may become more aware of the impact of their actions and the 
importance of communicating what they are doing and why. 



Effective: 

IP: When he was going through the checklist he said “It’ s 

in the other checklist.” Did you know what he meant? 
Talk about that. 

CA: I knew what you meant, but I just kept thinking “Are 
we on the right checklist?” 

FE: I knew we were on the right checklist, it’ s just that the 
two are somewhat redundant. But I figured as long as 
we got all of the items done. But I should have said 
something to you. 

CA: Okay. I guess I know that now, but I didn’t at the time. 



Effective: 

IP: You guys were very supportive in there. What did the 

Captain do to help create that environment? 

FO: Well, he set the tone right off the bat during the 

briefing, telling us that if we had anything at all to say, 
we should speak up. So that made it a lot easier to say 
what was on my mind when I needed to. 
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•f Ask crewmembers to discuss what they were each 
thinking. 

Encouraging crewmembers to openly articulate and discuss what 
they were thinking may help them understand each other’s point of 
view and thereby enhance communication. 

Effective: 

IP: CA, what did you do after you called for the engine fire 

checklist? 

CA: I got on the radio to talk to ATC. 

IP: And what did you think about that, FO? 

FO: I couldn’t believe you were talking to them. I thought 
we should have at least had the first couple of items on 
the checklist attacked by then. 

CA: And see, my initial gut feeling was “Get this thing 

headed for Boston now, whether you get the fire out or 
not, we need to be heading that way.” 

IP: Why do you think that CA? 



A: Analysis and Evaluation ofLOS Pejiojwance 

For crewmembers to learn from their LOS experience, it is 
essential that they analyze and evaluate what happened. Crews 
should analyze both what went well during their LOS and what did 
not work as well. The analysis must go beyond simply naming the 
strong and weak points of their performance. Crews can gain 
powerful insight by analyzing why things turned out the way they 
did, including factors that either enabled or hindered their success. 

/ Don’t give your analysis or Remember to refrain from giving your analysis until the crew have 

evaluation before the crew completed theirs, 
have completed theirs. 

Getting Crews to Evaluate their Performance 

It is important for crews to learn to critically evaluate their own 
performance in the LOS so they can carry this skill over into line 
operations. As the crew discusses the LOS, you may find it helpful 
to use a wallboard to list “strengths” and “areas for improvement” as 
they are identified by the crew. The following techniques may be 
useful in encouraging crews to evaluate their performance in depth. 

►F Get the crew to talk about what went well. 

Discussing what they did well helps the crew to recognize what 
strategies were effective in managing the LOS challenges and how 
these strategies might be used in line situations. This is also a good 
strategy to use when crews say everything went great during the 
LOS, so they can actively identify exactly what went well and why. 
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Effective: 

IP: FO, talk about one thing that went really well for you 

on that first leg. 

FO: I think the fact that I didn’t get overloaded at any time, 
that I was able to stay ahead of the airplane and because 
of that the rest of the leg went pretty smoothly. 

Ineffective: 

IP: You did a good job on that. Did you know you did a 

good job on that? 

FO: No. 

IP: Well, you did. 


■ Effective: 

• “What aspects of your performance were particularly strong 
on the first leg of your LOS flight?” 

• “What did you like about your performance on this trip?” 


Get the crew to talk about what could be improved, and 
how. 

Discussing what did not go well helps the crew identify 
problems that occurred, examine why they occurred, and determine 
how to resolve or avoid similar problems in the future. 

Effective: 

IP: What happened on your preparation for the approach; 

would you have done anything differently? 

FO: Well, I dropped the ball by not putting the approach in 
there. I don’t know where I was when that was 
happening. 

CA: Well, yeah, and I didn’t pick up on that either. We both 
missed it. 


Effective: 

IP: What did you think about your go around? 

CA: I was coming down and I just hit vertical speed and 
then max power and then flaps 15. 1 was trying to be 
smooth, but I thought it was really poor, actually. 


Effective: 

• “Are you satisfied with how you handled the generator 
failure? Why or why not?” 

• “Discuss one thing that you would like to have handled 
better during your climb out?” 

• “Why did the miscue occur, and what situation led to both 
of you missing it?” 

• “What events led to SOPs not being adhered to?” 
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►)- Troubleshooting: When the crew says everything 
went great. 

If, when asked to evaluate their performance, the crew- 
members say everything went great, facilitation may be required to 
encourage the crew to analyze and evaluate in more depth. If 
everything did go well in the LOS, encourage the crewmembers to 
discuss specific instances of good performance and analyze why 
they went as well as they did. Also, encourage them to discuss 
/ Encourage crew to discuss how they could have handled situations if they had not gone so 
how to handle things that well. If there were, in fact, situations that were not handled 

don t go well. effectively, draw the crew’s attention to a specific situation and ask 

if there is another way it could have been handled. Regardless of 
whether or not everything went smoothly in the LOS, it is 
important that the crewmembers understand both the factors that 
led to their successes and the factors that led to weaker aspects of 
their performance. 


Eliciting Deep Analysis 


y Push the crew to go 
beyond just describing 
what happened. 


To learn deeply from the LOS experience and take the lessons 
learned back to the line, crewmembers need insight into why 
events in the LOS turned out the way they did. Crewmembers can 
gain this insight through in-depth analysis of their LOS 
performance. You can help the crew analyze in depth by asking 
questions that require careful thought and detailed responses. 


»)• Ask questions that require description and analysis of 
LOS events. 

Open questions that require descriptive or analytical responses 
lead crewmembers to explore issues more thoroughly, which opens 
the door for deeper learning. 



Effective: 

• “Describe what happened there and why it happened the 
way it did.” 

• “What did you notice while watching that video segment of 
your LOS session?” 

• “Tell me what you did to handle that situation.” 

• “If you had to report to the safety people about this 
incident, how would you describe the situation and how 
you handled it?” 

Ineffective: 

Questions beginning with “Did you” or “Do you think” 

enable simple responses, and thus do not promote in-depth 

analysis. 

• “Did you follow the correct checklist for that problem?” 

• “Do you think you handled that situation effectively?” 
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Get the crew to analyze why they made the decisions 
they made. 

Articulating why they did what they did helps crewmembers 
gain insight into their decision-making processes, as well as the 
factors that influenced or should have influenced their decision 
making. 

Effective: 

IP: The RVR was down below twenty-four hundred, but 

you asked for two-twenty on the speed, and I was 
wondering why. 

CA: Well, the jet stream was real low, and there was all 
kinds of weather, with a warm front coming through, 
so I’m thinking that if we’re going to be hanging close 
we need to be clean maneuvering, so let’s add a couple 
initially to make it smooth but still slow it down. 




Effective: 

IP: Why did you turn off the pitot heat? 

CA: Because I looked at the outside air temperature and it 
was plus fifteen C, so I figured we were good with 
that. 



Effective: 

IP: Why did you decide to go back to Minneapolis instead 

of continuing on to Chicago? What was your thought 
process? 

CA: Well, I was thinkin g that we were still a lot closer to 
Minn eapolis than Chicago. Then again I also knew the 
weather was not as good in Minneapolis as it was in 
Chicago, so if that sick passenger was stable in the 
back, I was all for going on to Chicago. But as he 
started to deteriorate rapidly, I thought the best course 
of action would be to turn around and go back because 
I thought the guy might die if I didn’t get him back 
pretty quick. 



Effective: 

• “Why did you feel it was necessary to disregard ATC until 
you completed your checklist?” 

• “What made you decide to verify the information you 
received from ATC?” 
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When you analyze for the crew it leaves little for them to say. It 
also gives them the message that you are teaching them so they are 
not expected to analyze for themselves. 

Ineffective: 

• ‘The reason you got into trouble during the approach is that 
you didn’t communicate to the CA that you were unsure of 
the clearance.” 



+ Get crewmembers to discuss what they were thinking. 

Getting the crew to discuss what they were thinking during the 
LOS can help them discover what information and events influenced 
their actions. 



Effective: 

IP: What was going through your mind at that moment? 

CA: Well, I was thinking about the sick lady in the back and 
that we had to get down and get her to a doctor, but 
you can’t rush the approach because we might have to 
make a missed approach and then it would take even 
longer. 


ip!! Effective: 

S • “What were you thinking when you received the traffic 
M alert?” 


*>• Encourage the crew to discuss the factors that 
enabled or impeded their success. 

Identifying underlying factors that enabled or impeded their 
success in the LOS can help the crewmembers recognize similar 
factors when they occur on the line. 

Effective: 

IP: CA, was there something that helped make that 

windshear recovery go so well? 

CA: Yes, FO was very assertively making speed callouts all 
the way that helped me stay on target. 




m* 


Effective: 

IP: CA, can you think of any factors that contributed to your 

getting rushed on that approach? 

CA: Well, in hindsight, I should never have accepted the turn 
onto base leg before we had finished the abnormal checklist. 
FO: I wasn’t sure if you realized we had not finished it. I wish 
I had queried you on that. 
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L: Line Operations - Applying Lessons from LOS 

To help crewmembers transfer the lessons they learn in the LOS 
to the line, encourage them to discuss how LOS performance and 
associated CRM issues relate to effective line operations. 

Get the Crew to Discuss Related Line Incidents 

Getting the crewmembers to discuss actual line incidents and 
accidents related to CRM issues that arose during the LOS can help 
them to appreciate the role of CRM in line operations. For example, 
if a crew communicated well during the LOS which resulted in a 
positive outcome, the importance of their effective communication 
can be reinforced by having them discuss a line accident that resulted 
from poor communication. 

I Effective: 

IP: Looking back on the [XXX] accident we had where the 

guys taxied onto the wrong runway, what was the 
major problem there? 

CA: Lack of com muni cation. And not paying attention. 

When you’re taxiing under conditions like that both 
guys better keep their heads where they should be. 

They should be thinking about nothing but taxiing. 


Get the Crew to Discuss How to Apply Their 
Success to Line Operations 

The next step in getting the crewmembers to transfer what they 
leam in the LOS to line operations is to get them to explore how they 
can apply techniques they utilized successfully in the LOS to 
overcome obstacles on the line. Getting crewmembers to talk about 
how they would handle difficult situations enables them to develop 
effective, pre-planned strategies for dealing with real situations 
when they occur on the line. For example, one crew’s success in a 
particular LOS might have been aided by the fact that both 
crewmembers were good at explicit sharing of appropriate 
information. In the debriefing, you might find it useful to facilitate a 
discussion of how to deal with crewmembers on the line who are 
not as good at sharing information. 

I Effective: 

• “What would you do if you ran into one of those guys on 
the line?” 

• “How would you handle the same situation if the 
CA/FO/FE was completely non- communicative?” 

• “In what other situations could you use that technique?” 
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Get Crew to Discuss What They Would do 
Differently 

Discussing what they would do differently enables 
crewmembers to develop strategies they can use to make more 
effective decisions and avoid similar incidents in line operations. 
Ideally, crewmembers should discuss the strategies they can use to 
turn each item in their areas for improvement list into strengths. It is 
often useful to frame this discussion in terms of how the 
crewmembers can handle similar situations if they occur on the line. 



Effective: 

IP: CA, tell me what you could have done to make the 

approach onto runway three- two work out better than 
it did. 

CA: I could have just immediately gone to the approach 
page and put in runway three-two, brought it up and it 
would have probably had Grunds. Even if it hadn’t, it 
would have had everything else we needed, and things 
would have gone a lot more smoothly. 

Ineffective: 

IP: You could have avoided that by using your FO more 

effectively. 

[Telling the crew what they should have done differently 

denies them the opportunity to figure it out for themselves.] 


Effective: 

• “What CRM techniques can you use to keep from getting 
overloaded in the future?” 

• “How will you manage that situation if it comes up 
tomorrow on the line?” 


Get the Crew to Discuss How They Will Do Things 
Differently on the Line based on Their Experience 
in the LOS 

This final step requires crews to specifically tie what they have 
learned in the debriefing to the line. Having crews discuss how they 
will perform differently based on what they have learned can help 
them make the connection necessary to transfer thoughts into 
actions. 

Effective: 

• “What would you do differently back on the line to avoid 
the problems that occurred during the second leg?” 

• “Let’s say you’re flying into LAX tomorrow, and ATC 
switches the runway on you at the last minute. What CRM 
techniques can you use to manage the situation?” 
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Organizing the Discussion: 
The C-A-L Model 
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Organizing the Discussion: 
The C-A-L Model 


The C-A-L Model incorporates three main concepts: CRM (C), Analysis and evaluation of LOS 
performance (A), and application to Line operations (L). This model provides a simple framework 
for crew discussion of each LOS situation or topic. 
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Guidelines for 

Facilitating LOS Debriefings 


Facilitation Basics 

y Keep the discussion crew-centered. 

y Encourage crewmembers to participate actively and do most of the talking. 

•/ Adapt the level of facilitation you use to the capabilities of each crew. 

/ Balance your dual role as instructor and facilitator. 
y Reinforce good crew performance following crew analysis. 

/ Show by your attitude that you are interested in what the crewmembers have to say. 

/ Don’t lecture or make long speeches. 

/ Don’t interrupt or leave a topic while the crew still has something to say. 
y Use the highest level of facilitation possible to maximize crew self-discovery. 

/ Adapt the level of facilitation to accommodate varying crew needs throughout the session. 

The Introduction 

y Clarify your role as the instructor and detail your expectations for crew participation. 

/ Provide a persuasive rationale for why the debriefing should be crew-centered. 

/ Tell the crew how long the session will last. 

/ Don’t cut sessions short for high-performing crews. 

Agenda and Format 

y Explain the format the debriefing will follow. 

/ Help the crew develop an agenda. 

/ Ensure that all critical issues are covered 

y Use the C-A-L Model as a framework for discussion of each topic. 

Facilitation Techniques 

y Use questions to promote in-depth crew participation. 

/ Follow up on crew topics and redirect crew questions and comments back to them. 
y Ask questions that begin with what, how, and why to encourage deeper discussion. 

/ Encourage crewmembers to discover their own answers. 
y Make sure all crewmembers are fully drawn into the discussion. 
y Direct questions to quiet crewmembers. 

y Ask quiet crewmembers to comment on what other crewmembers said. 
y Avoid unwittingly hindering crew participation. 

y Don’t answer for the crew when they don't immediately respond to your question. 

/ Re- word questions rather than giving the answer. 

/ Use active listening to encourage continued participation. 

/ Use silence/pauses to elicit thoughtful crew responses. 
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Use of video 

✓" Index important events during the LOS. 
y Don’t show video segments you don't intend to discuss. 
y Learn how to use the video equipment efficiently. 

/ Introduce each video segment and remind the crew that they will analyze what they see. 

/ Pause the video for comments and to discuss important aspects of crew performance. 

Reinforcing CRM through crew interaction 

y Encourage crewmembers to address each other directly. 

/ Ask crewmembers to discuss how they were affected by each other’s actions. 
y Encourage crew to discuss what they were each thinking. 

Eliciting in-depth analysis and evaluation 

y Don’t give your analysis or evaluation before crew have completed theirs. 

/ Get crew to talk about what went well. 

/ Get crew to talk about what could be improved, and how. 

y Encourage crew to discuss how they might have handled things if they had not gone so well. 
y Push the crew to go beyond just describing what happened. 
y Ask follow-up questions that require in-depth analysis. 

✓ Ask crew to analyze why they made the decisions they made. 

/ Get crew to discuss what they were thinking. 

y Encourage the crew to discuss the factors that enabled or impeded their success. 
y Have the crew discuss how they can apply what they learned in the LOS during line operations. 

Summary 

y Briefly summarize the debrief: Reiterate important issues and tie LOS events to line operations. 
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